Modeling Perceptional Fluency with Visual Representations

Visual representations are ubiquitous instructional tools in science, technology, engineering, and math (STEM) domains. The goal of our ongoing research is to develop a new methodology for cognitive modeling of perceptual learning processes so as to create adaptive technologies that support perceptual fluency. We are using metric learning methods to assess which visual features novice students and experts focus on when presented with visual representations. Comparing novice to expert perceptions will establish which visual features perceptual support should help students attend to (e.g., because experts focus on them but novices do not). Hence, metric learning will provide a skill model of student perceptions (i.e., analogous to what verbalization techniques provide in traditional cognitive modeling). In this paper, we apply metric learning to identify salient features in the visual perception of molecular diagrams used in chemistry education. 1. Visual Representations in Chemistry Instructors use the visual representations shown in Figure 1 to help students learn chemical bonding. Yet, to a novice student, these visual representations may not be helpful because the student may not know how to interpret the representations. First, they typically focus on one set of representational competencies: students’ conceptual understanding of representations (e.g., the ability to explain how visual features depict concepts). This focus mimics education psychology research’s focus on conceptual learning (Ainsworth, 2006; Seufert, 2003). However, research suggests a second type of representational competency is crucial for students’ learning success: perceptual knowledge (Kellman & Massey, 2013; Massey et al., 2011), the ability to rapidly and effortlessly perceive information based Proceedings of the 33 rd International Conference on Machine Learning, New York, NY, USA, 2016. JMLR: W&CP volume 48. Copyright 2016 by the author(s). on visual features of the representations. This ability results from implicit forms of learning. For example, expert chemists simply ’see’ that the molecules depicted in Figure 1 have a local negative charge by the Oxygen atom, without having to make a conceptual inference. In contrast, novice students may wonder: does the red color in the ball-and-stick figure (Figure 1-b) mean the same thing as in the electrostatic potential map (EPM; Figure 1-d)? (It does not.) Instructors often ask students to use visual representations that they have never seen before to make sense of concepts that they have not yet learned about (Airey & Linder, 2009; Wertsch & Kazak, 2011), an issue known as the representation dilemma (Dreher & Kuntze, 2015). Hence, to succeed in STEM, students need representation skills that enable them to use visual representations to make sense of and solve domain-relevant problems (Ainsworth, 2006; Gilbert, 2005). Figure 1. Representations of water. a: Lewis structure; b: balland-stick figure; c: space-filling model; d: electrostatic potential